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Abstract. The scientific research proposal is part of the task to be carried out in academic and research institutions around the 
world. This is a complex decision-making problem, because decision-makers must determine the projects that are appropriate to 
the subjects addressed by the institution, those projects must be achievable within a reasonable deadline, they must have the 
financial means and the budget necessary to be carried out, the staff must be sufficiently qualified and an optimum number of 
personnel must be available to succeed the tasks and not interfere with other research projects. This is a predictive problem, thus, 
the proposed model is based on Delphi method for evaluating research projects and is supported by neutrosophy. Delphi method 
is widely applied in the prediction of future events, in this model we introduce the uncertainty and indeterminacy modeled with 
neutrosophy. As the best of our knowledge, this model is the first one, which applies a neutrosophic Delphi method in the 
evaluation of scientific research proposals. Finally, a hypothetical case study illustrates the applicability of the method. 


Keywords: Research proposal, Delphi method, fuzzy Delphi method, single valued neutrosophic set, single valued triangular 
neutrosophic number. 


1 Introduction 


A research proposal is a document that proposes a research project, usually in science or academia, and that 
usually constitutes a request for sponsorship of such research, see [1-10]. The proposals are evaluated on the cost 
and potential impact of the research, and on the robustness of the proposed plan to carry it out. Generally, research 
proposals address several critical points, including the following: 

e Which research questions will be addressed and how they will be addressed, 
How much time and expenses will be needed for research, 
What previous research has been done on the subject, 
How research results will be evaluated, 
How research will benefit the sponsoring organization and other parties. 

Research proposals could be requested, which means that they are sent in response to a request with specific 
requirements, such as a request for proposals, or may be unsolicited, which means that they are sent without prior 
request. Other types of proposals include "pre-proposals", where a letter of intent or documentary summary is sent 
for review prior to the submission of a full proposal; follow-up proposals, which reiterate an original proposal and 
its funding requirements to ensure continued funding; and proposals for renewal, which seek the continued 
sponsorship of a project that would otherwise be terminated. 

Academic research proposals are usually written as part of the initial requirements for writing a thesis, research 
work or dissertation. In general, they follow the same format as a research work, with an introduction, a review of 
literature, a discussion of the methodology and objectives of research, and a conclusion. This basic structure can 
vary between projects and between fields, each of which may have its own requirements. 

The scientific method is a methodology for obtaining new knowledge, which has historically characterized 
science, consisting of systematic observation, measurement, experimentation, and the formulation, analysis and 
modification of hypotheses. Other characteristics of the scientific method are deduction, induction, abduction, 
prediction, falsifiability, the reproducibility and repeatability of the results, and the peer review. The rules and 
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principles of the scientific method seek to minimize the influence of the subjectivity of the scientist in his\her 
work, which reinforces the validity of the results, and therefore of the knowledge obtained. 

The selection of the most appropriate research topic in the academic or the research institution is not a trivial 
problem, it needs of the assessment of the topic relevance in the near future, that is, it is a predictive problem. It is 
also complex, since the decision depends on different factors, some of them depend on the institution’s researchers 
and others are external. This complexity of the problem requires of the experts’ opinion on the subject, rather than 
measuring with objective indicators. The experts are those who can carry out the selection of the most promising 
projects, which give visibility to the institution and at the same time those projects must be achievable within a 
reasonable time. Other aspects to be considered are that researchers must have the capacity to attain the selected 
projects, that there exists the optimal number of scientific personnel working on the project, that the institution 
must have the necessary financial support to accomplish the research, and the project must be sufficiently relevant 
such that it can be published in high-impact scientific journals in a relatively short period of time or it results in 
patents, palpable economic and social results, among others. 

Due to the problem complexity and the large number of variables to consider, the proposal selection contains 
elements of uncertainty and at the also experts could have doubts, ignorance, inconsistencies, among other 
elements. 

This paper aims to propose a model for research proposals selection and evaluation. This model is based on 
the Delphi method, which is used in predicting future scenarios or events through expert assessment, see [11, 12]. 
Basically Delphi method is based on the intuitive idea that a group of experts will come to better conclusions than 
only one of them. The Delphi method consists of applying questionnaires to a group of experts, anonymously, and 
then each of them gives a response in a first round. The index of agreement between experts is then calculated 
using a central tendency statistical measure, and if the agreement is not sufficient, a second round is conducted for 
the experts to reconsider their assessments and so on until sufficient consensus is reached among them. One 
criticism of the method is that it can converge very slowly and therefore some experts may not continue to 
collaborate, nevertheless this is a widely used method. 

Other authors have extended this method into uncertainty environments, for example fuzzy Delphi includes 
uncertainty and represents it in form of fuzzy sets, in particular fuzzy numbers are used, see [13-17]. Ishikawa et 
al. in [18] propose a fuzzy Delphi method where a survey is designed in such a way that a single round is sufficient 
to perform the calculations. In general, the fuzzy Delphi method has application in several real problems, see [13- 
17]. 

Other approaches are based on neutrosophy, which generalizes fuzzy sets, fuzzy intuitionist sets, among others. 
In the context of neutrosophy Delphi method takes into account the neutrality given by contradictions, ignorance, 
inconsistencies, among other ones, typical of decision-making. Some papers model fuzzy Delphi method into a 
neutrosophic framework, see [19, 20]. Abdel-Basset et al. use Delphi method combined with AHP, in a 
neutrosorphic environment, see [21]. 

The model proposed in this paper is based on the Delphi method, which helps to select a set of scientific 
research proposals in a neutrosophic environment. We have not found in the consulted literature the use of Delphi 
method applied in this topic in a neutrosophic environment. This neutrosophic Delphi method uses single value 
triangular neutrosophic number, see [22]. The method takes advantage of the possibility of evaluating research 
proposals in form of linguistic terms, in addition to considering the uncertainty and indeterminacy inherent to 
neutrosophy frameworks. It is a decision-making model because it allows the evaluation of project alternatives by 
criteria. In addition, we explicitly set out the minimum criteria that should be considered in conducting evaluations. 

The paper consists of the following structure; after this introduction follows Section 2 which contains the 
concepts necessary to design the model, such as the basic concepts of neutrosophy, its aggregation operators, 
among others, as well as a brief explanation of the Delphi method. Section 3 describes the proposed model and 
provides an illustrative case study of the application of the model in a real-life problem. The paper finishes with 
the conclusions. 


2 Basic concepts 


This section discusses the concepts and methods to be used throughout this article. Section 2.1. contains a brief 
explanation of the classic Delphi method, whereas Section 2.2. contains the main concepts of neutrosophy, among 
them we can find, neutrosophic sets, single valued neutrosophic sets, single valued triangular neutrosophic 
numbers, aggregation operators for single valued triangular neutrosophic numbers, among other concepts of 
interest. 


2.1 The Delphi method 


The Delphi method is a structured communication technique, which is developed as an interactive systematic 
prediction method, based on a panel of experts, see [11, 12]. It aims to achieve a consensus based on discussion 
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among experts. It is a repetitive process, where its operation is based on the elaboration of a questionnaire to be 
answered by the experts. Once the information is received, another questionnaire based on the previous one is re- 
performed to be answered again. 

Finally, the study will draw its conclusions from the statistical analysis of the obtained data. 

The Delphi as a methodology of forecasting uses expert judgments in technology or social processes 
considering the responses to a questionnaire to examine the likely guidelines for the development of specific 
technologies, meta-types of technologies or different processes of social change. The summary of expert judgments 
(in the forms of quantitative assessments and written comments) are provided as feedback to the experts themselves 
as parts of a next round of questionnaire. Experts then reassess their views in the light of this information, and a 
group consensus tends to emerge. The Delphi technique is based on firm concepts to draw conclusions with 
supported arguments. 

Delphi is based on: 

e Anonymity of participants. 

e Repeatability and controlled feedback. 

e Group response in statistical form. 

Before starting Delphi, a number of previous tasks are performed, such as: 

e Define the context and time horizon in which the forecast on the subject under study is to be made. 

e Select the panel of experts and get the commitment to collaboration. People who are elected should not 

only be very knowledgeable about the subject on which the study is being conducted, but should present 
a plurality in their approaches. This plurality should avoid the appearance of biases in the information 
available in the panel. 

e Explain to experts what the method is. This is intended to get obtaining reliable forecasts, because the 
experts are going to know at all times what is the objective of each of the processes required by the 
methodology. 

The core of Delphi technique is a series of questionnaires. The first questionnaire may include general 
questions. At each later stage, the questions become more specific because they are formed with the answers to 
the previous questionnaire. 

The Delphi technique comprises at least three phases: 

1. A questionnaire is sent to a group of experts. 

2. Asummary of the first phase is prepared. 

3. Asummary of the second phase is prepared. 

Three phases are usually recommended, but more phases can be used, as in the safety management Delphi 
study. 

The number of experts involved can range from just a few to more than 100, depending on the scope of the 
issue. A range of 15-30 is recommended for a focal issue. As long as experts participate, the costs as well as the 
coordination required for the technique will also be raised. 


2.2 Basic concepts of neutrosophy 


Neutrosophy is a branch of philosophy that studies the origin, nature and scope of neutralities, as well as their 
interactions with different ideological spectra. In mathematics and logic, the most important concept is the 
neutrosorphic set that generalizes the fuzzy sets of Zadeh and the fuzzy intuitionist sets of Atanassov, in the 
following these definitions are formally defined. 


Definition 1: ((22]) The Neutrosophic set N is characterized by three membership functions, which are the 
truth-membership function Ta, indeterminacy-membership function I,, and falsity-membership function Fa, where 
U is the Universe of Discourse and Vx eU,TA(x),IA(x), FA(x) c]-—0,1+[,and — 0<inf TA(x) + 
inf IA (x) + inf FA (x) ssup TA(x) + sup IA (x) + sup FA (x)3 +. 

See that according to Definition 3, T(x), Ia(x), and Fa(x) are real standard or non-standard subsets of | — 
0,1+ [and hence, TA(x), IA(x) and FA(x) can be subintervals of [0,1]. 

Definition 2: ((22]) The Single-Valued Neutrosophic Set (SVNS) N over U is A = {< 
x; TA(x), IA(x), FA(x) > : x €U}, where TA: U-[0, 1], IA: U->[0, 1], and FA: U->[0,1],0 sTA(x) + 
TA(x) + FA(x) $3. 

The Single-Valued Neutrosophic number (SVNN) is symbolized by N = (t,i, f ), such that 0 <t,i, f <1 and 
Ost +i + fs3. 

Definition 3: ((22]) The single-valued triangular neutrosophic number 4 = ((a4,a2-43); M3, B3, Ya), iS a 
neutrosophic set on R, whose truth, indeterminacy and falsity membership functions are defined as follows, 
respectively: 
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TsO) = ot ane (1) 

[ a a) a2<xSa3 
0, otherwise 
ao>—-X+ X-a 

((22—x+ Bax a) 2 Bal wd) a, SxSa, 

| a,— ay 

X=a 
Ix(x) a 5 

X-a,t a,—X 

pa 2 Bat 3 (=a, 4+Bal@s—%)) an <x <a; (2) 

az 
1, otherwise 
iGxer uae), eres 
[| aga ay ? 
F,(x) = =) X= a, 
a X-a,t+ a, —X (3) 
_—— 2 ue (x= a2 + val@s— 9) a, <x Sa; 
a2 
otherwise 


Where o3,B3,Ys € [0,1], a,, az,a3; € Randa, <a, < a3. 


Definition 4: ({22]) Given 4 = ((a,,a2,a3); 0g, B3,Y4) and b = ((bj,b,, bs); ag, Bg, Vg) two single-valued 
triangular neutrosophic numbers and X any non null number in the real line. Then, the following operations are 


defined: 


19. Addition: 4 +b = ((a, + b,,a, + bz,a3 + bs); a3 AO, Bs V Bs, Ya V Ys) 
20. Subtraction: d— b = ((a, —b3,a, — b,a3 —b,); a3 A as, Bx V Bg, Ya V Vp) 
21. Inversion: 4-1 = ((a371,a,71,a,71); a5, Bg, Ys), where a,,a,a3 # 0. 
22. Multiplication by a scalar number: 
aes fea eocaa O3,Bs,Ya), A>O 
((Aaz,Aaz,Aa;)iq,BaYa)  A<0 
23. Division of two triangular neutrosophic numbers: 


f a, a2 a3 
({— 33 A Og, Bs V Bs, Ya V ¥p), a3 > O and bz > 0 
| b;’ by "Bi 


a3 az ay 
=4(2, by a ax A ag, Ba V Bs, Ya V ¥5), a3 < O and b; > 0 
3 


ar ae 
(F by’ Ds =); ax A og, Ba V Bg, Ya V Yg), a3 < O and b3 <0 
24. Multiplication "Of two triangular neutrosophic numbers: 
- ((a,b,,a,b2, a3b3); 03 A ag, By VBs,Ya V¥5) a3 > Oandbz >0 
ab = {((a,b3,a2b2,a3b,); a3 A og, Ba V Bp, Ya V Yo), az <Oandb; >0 
((agb3, a,bz,a,b,); ag A ag, Ba V Bs, Ya V Vp), a; < 0Oandb, <0 
Where, A is a t-norm and V is a t-conorm. 
Let 4 = ((a4,a2,43); M3, Bs, Yx) be a single valued triangular neutrosophic number, then, 


a Or 


1 4 
S(@) = gla: +a2 + ag](2 + o-Ps — Ya) . 
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(5) 


AA) = 5 lay + ay + ag](2-+ a4-Bs + Ys) 


They are called the score and accuracy degrees of 4, respectively. 


Let {A,,A2, Ag} be a set of n SVTNNs, where Aj = ((aj,b;, Gj); aa, Ba,» Ya;) (j = 1, 2, ..., n), then the 
weighted mean of the SVTNNs is calculated with the following Equation: 


a (6) 


Where 4, is the weight of Aj, A;<[0, 1] and yj=14j = 1. 


3 The Delphi model in the neutrosophic environment 


This section is dedicated to describe the model proposed in this paper. Let us start with two tables, Tables 1 
and 2. Table | contains the scale for measuring the weights of the criteria and Table 2 summarizes the scale of 
evaluations associated with the single-valued triangular neutrosophic numbers (SVNTN). We finish with a 
hypothetical case study. 


3.1 The method 


Linguistic terms SVTNN 

Extremely unimportant (EU) ((0,0, 1); 0.00, 1.00, 1.00) 
Not very important (NVI) ((0, 1, 3); 0.17, 0.85, 0.83) 
Not important (NI) ((1, 3,5); 0.33, 0.75, 0.67) 
Medium (M) ((3, 5,7); 0.50, 0.50, 0.50) 
Important (1) ((5, 7,9); 0.67, 0.25, 0.33) 
Very important (VI) ((7, 9, 10); 0.83, 0.15, 0.17) 
Extremely important (ED) ((9, 10, 10); 1.00, 0.00, 0.00) 


Table 1. Importance weight as linguistic variables and their associated SVTNN. 


Linguistic term SVTNN 

Very low (VL) ((0,0, 1); 0.00, 1.00, 1.00) 
Medium low (ML) ((0, 1,3); 0.17, 0.85, 0.83) 
Low (L) (C1, 3,5); 0.33, 0.75, 0.67) 
Medium(M) ((3, 5,7); 0.50, 0.50, 0.50) 
High (H) ((5, 7,9); 0.67, 0.25, 0.33) 
Medium high (MH) ((7,9, 10); 0.83, 0.15, 0.17) 
Very high (VH) ((9,10, 10); 0.00, 1.00, 1.00) 


Table 2: Linguistic terms for evaluations associated with SVTNN. 


Let us observe two important aspects, which are the following: 

1. The scales shown in Tables | and 2 are inspired by the linguistic scales in [16]. SVTNNs are obtained by 
rescaling the original 0-1 scale to a 0-10 scale. The values a3, {83,3 are adapted from another scale appeared 
in [16]. 

2. The scale shown in Table 2 was linguistically taken in such a way, because the survey questions asked to 
experts will be done in the form of probability of events occurrence, which will be evaluated linguistically as 
Low, Medium low, etc. 
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Let us observe that the values obtained above are more accurate than fuzzy numbers, because they contain 
more elements; not only the belongingness, but also the non-belongingness and the indeterminacy. 
The algorithm for evaluating research proposals that we offer is as follows: 
1. Starts from a subject or group of subjects that are usually investigated in the institution. 
2. Experts on the proposed subject or subjects are selected to evaluate the projects and at least one moderator. 
The experts will be denoted by Ej, Eo, ..., En. 
Usually each academic or research institution has a group of specialists who are part of the scientific 
council that is where the scientific projects of the institution are discussed. This group of people could be 
used to carry out the evaluations, although external experts are also useful. Experts need not to be in touch 
with each other, so the moderator must design, implement and process the surveys. 
3. Then experts are asked to propose projects that can serve as a research proposal on the basis of the topic 
identified in the previous point. Each of them proposes at least one, they are called pj, po, ..., Pm- 
4. Experts could be asked to identify the criteria they consider for evaluating projects. However we suggest 
the following criteria: 
C,: The project is a sufficiently relevant scientific contribution to the subject being investigated over a 
sufficiently long period of time. 
C»: The project is scientifically achievable in a sufficiently short time. 
C;: The institution has sufficient qualified staff to carry out the project. 
C4: Sufficient personnel are available to conduct the investigation. 
Cs: There are the means and the budget necessary to carry out the research. 
Co: The desired results will be obtained (publications in high-impact journals, patents, discussion of 
Master's or doctoral theses, solution of a real-life problem, etc.) in a reasonable time. 
C;: The project serves as a basis or starting point for another project. 
Cs: The project gives scientific visibility, economic income, prestige, etc. to the institution. 
The survey can contain questions as follows: 
4.1. What do you think is the probability that project P will become a sufficiently relevant scientific 
contribution to the subject under investigation over a sufficiently long period of time? 
4.2. What do you think is the probability that project P will be scientifically achievable in a sufficiently 
short time? 
4.3. What do you think is the probability that the institution will have sufficiently qualified staff to carry 
out project P? 
4.4. What do you think is the probability that the institution will have sufficient available staff to carry 
out project P? 
4.5. What do you think is the probability that the institution will have the means and the budget to carry 
out project P? 
4.6. What do you think is the probability that the desired results of project P will be achieved in a 
reasonable time? 
4.7. What do you think is the probability that project P will serve as a basis for another project? 
4.8. What do you think is the probability that project P will provide scientific visibility, economic income, 
prestige, etc. to the institution? 
The answers will be given on the basis of the linguistic scale shown in Table 2. 
Each of the experts assesses the importance of each of the criteria. ,; shall denote the linguistic value 
according to Table 1, which expert Ej associates with the criterion Cj @ = 1, 2,...,n; j= 1, 2, ..., 8). 
Wij = A(W;;) a= 1, 2,...,n;j =1, 2, ..., 8) is calculated using formula 5. Then they are normalized with 


; Wis 
respect to each expert, let us use the notation W;; = = = : 
i=1 Wij 

5. Each expert Ej (i = 1, 2,..., n) evaluates each project px (k = 1, 2,..., m) with respect to criteria Cj j = 


1,2,..,8). 

6. For each expert Ej G = 1, 2,..., n), the evaluation of each project p, (k = 1, 2,..., m) is obtained by 
aggregating their values by criterion using the W;; weights in Formula 6. So we have an evaluation of 
each expert for each project. Let us denote by Pi, the evaluation of the k" project by the i” expert in form 
of the SVTNN associated with the linguistic term in Table 2. 


1 
dizi Pik 
n 


7. Itis calculated P, = which is the mean of the evaluation of each project for all experts. 


Tr 7 = D. 
8. The Consensus Indexes for each project p;, are calculated with formula CI, = MiaslAP iAP DI 


n 
9. If Cl, < 0.2, see [16], then there exists sufficient expert consensus for all projects and go to Step 11, 
otherwise there is no consensus and go to point 10. 
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10. The moderator anonymously informs each of the experts about the results. He\she asks for explanations 
for each of them, including the weights assigned by them to the criteria and go to a next round. 
Emphasis is placed on those projects that reached a consensus index of CI, > 0.2, which reduces the 
algorithm complexity when concentrating recalculation only on those projects where there was not 
satisfactory consensus. Next go to point 5. 

11. P, = A(P,) are calculated according to Equation 5. P, is ordered, where projects with higher values are 
preferred over those with lower values. 

Finish. 


3.2 Case Study of case: Comparative analysis 


A research group of an academic institution has as its research topic artificial intelligence applied to digital 
image processing. The institution wishes to work on new projects on the subject so that they become doctoral 
theses of some members. Supervisors wish to determine which projects could be approved to obtain doctoral theses 
froma group of members within a maximum of five years. To this end, they decide to apply the method we propose 
in this article as follows: 

1. They decided that the general theses themes should be “artificial intelligence applied to digital image 

processing”. 

2. They select one moderator within the institution. The moderator selects the panel of experts on the subject 
to carry out evaluations. Five experts were selected; let us denote them by E), E», Es, E4, Es. None of them 
knows the identity of the others, which is why the moderator keeps in touch with each one via email. Any 
queries that experts have to make about the institution's data are directly asked to the moderator. 

3. The moderator consults them to ask for proposed projects on the subject. These were four; let us call them 
Pi, P2, P3, pa. This consultation process remains anonymous. 

4. The moderator distributes the survey with the questions in Step 4 of the algorithm and asks them to 
evaluate the importance of each of the given criteria on the linguistic scale in Table 1. 

The results were as follows as in Table 3 for linguistic evaluations and Table 4 for the crisp values of the 
normalized weights of evaluations in Table 3: 


Criteria\Weight given by: Ei E2 K3 Ka Es 

Ci I VI I M I 

CQ VI I I VI I 

C3 VI VI M I VI 

C4 I VI I I VI 

Cs VI I VI I I 

Co M I M I M 

C7 M M M I I 

Cs I I M EI I 

Table 3: Importance given by experts to the criteria in form of linguistic terms. 

Criteria\Weight given by: Ei E2 K3 E4 Es 

Ci 0.122729 0.150978 0.145338 0.076947 0.121720 
CQ 0.157475 0.117665 0.145338 0.152047 0.121720 
C3 0.157475 0.150978 0.094375 0.118498 0.156181 
C4 0.122729 0.150978 0.145338 0.118498 0.156181 
Cs 0.157475 0.117665 0.186485 0.118498 0.121720 
Co 0.079694 0.117665 0.094375 0.118498 0.121720 
C7 0.079694 0.076406 0.094375 0.118498 0.121720 
Cs 0.122729 0.117665 0.094375 0.178516 0.121720 


Table 4: Importance given by experts to the criteria in form of normalized crisp values. 


5. Each expert evaluates each project on the basis of the criteria. The results are given in Tables 5-9. 


Criteria\Project: pi p2 p3 P4 
Ci H M L H 
C2 M M VH M 
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C3 L ML H L 
C4 L VL L H 
Cs L M H VL 
Co M ML H L 
C7 L M VH L 
Cs L ML VH VL 
Table 5: Projects evaluated per criterion by Expert 1. 

Criteria\Project: pi p2 p3 P4 
Ci VH ML M MH 
C2 ML H H H 
C3 L L MH ML 
Cs M ML M MH 
Cs L H MH L 
Co MH L H VL 
C7 M VL H VL 
Cs M L VH L 
Table 6: Projects evaluated per criterion by Expert 2 

Criteria\Project: pl p2 p3 P4 
Ci M VL VL VH 
C2 L ML M H 
C3 M L M M 
Cs VL ML ML MH 
Cs M H MH L 
Co M L MH VL 
Cr M MH H VL 
Cs M VL H L 
Table 7: Projects evaluated per criterion by Expert 3. 

Criteria\Project: pl p2 p3 P4 
Ci MH MH VL MH 
C2 L MH H MH 
C3 ML L MH VL 
Cs ML L ML MH 
Cs VL ML VH L 
Co VL L MH VL 
C7 ML H H ML 
Cs ML L MH VL 
Table 8: Projects evaluated per criterion by Expert 4. 

Criteria\Project: pi p2 p3 P4 
Ci M ML ML VH 
C2 H H H VH 
C3 ML L VH VL 
Cs ML ML M M 
Cs VL L M L 
Co VL L MH VL 
Cr VL H H VL 
Cs L VL H L 


Table 9: Projects evaluated per criterion by Expert 5. 
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6. Table 10 contains the project evaluation by each expert after aggregating the set of projects, using the 
criteria weights of Table 4 in form of SVTNN. 


Expert\Project p1 


p2 


p3 


P4 


Ei ((1.97,3.97, 5.97); ((1.55,2.95, 4.82); (G.46,7.10, 8.38); ((2.02,3.46, 5.18); 
0.33, 0.75, 0.67) 0.00, 1.00, 1.00) 0.33, 0.75, 0.67) 0.00, 1.00, 1.00) 
E2 ((3.49,5.22, 6.80); ((1.56,3.11, 5.03); ((5.40,7.29, 8.78); ((2.94,4.40, 5.90); 
0.17, 0.85, 0.83) 0.00, 1.00, 1.00) 0.50, 0.50, 0.50) 0.00, 1.00, 1.00) 
E3 ((2.27,3.98, 5.84); ((1.78,3.01, 4.68); ((3.63,5.19, 6.77); ((3.62,5.09, 6.47); 
0.00, 1.00, 1.00) 0.00, 1.00, 1.00) 0.00, 1.00, 1.00) 0.00, 1.00, 1.00) 
E4 ((0.69,1.68, 3.37); ((2.73,4.61, 6.38); ((5.33,6.94, 8.21); ((2.55,3.60, 4.84); 
0.00, 1.00, 1.00) 0.17, 0.85, 0.83) 0.00, 1.00, 1.00) 0.00, 1.00, 1.00) 
Es ((1.10,2.14, 3.82); ((1.57,3.05, 4.93); ((4.62,6.34, 7.95); ((2.90,3.95, 5.10); 


0.00, 1.00, 1.00) 


0.00, 1.00, 1.00) 


0.17, 0.85, 0.83) 


0.00, 1.00, 1.00) 


Table 10: Projects evaluated by Experts. 


7. From Table 10 it is obtained  P, = ((1.904,3.398,5.16);0.00,1.00,1.00), _P, 
(( 1.838, 3.346, 5.168); 0.00,1.00,1.00), P, = ((4.888, 6.572, 8.018); 0.00,1.00,1.00), and P, 
((2.806, 4.100, 5.498); 0.00, 1.00,1.00). In addition we have P, = A(P,) = 2.6155, P, = A(P,) 
2.5880, P; = A(P;) = 4.8695, and P, = A(P,) = 3.1010. 

8. Table 11 contains the crisp values by applying the accuracy function to the values in Table 10. 


Expert\Project pi p2 p3 Pa 

Ei 3.3497 2.3300 5.8894 2.6650 
E2 4.1683 2.4250 6.7094 3.3100 
E3 3.0225 2.3675 3.8975 3.7950 
E4 1.4350 3.6872 5.1200 2.7475 
Es 1.7650 2.3875 5.0821 2.9875 


Table 11: Projects evaluated by Experts in form of crisp values. 


Therefore, the Consensus Indexes for the projects are the following: 
Cl, = 0.945, CI, = 0.38824, Cl, = 0.85898, and Cl, = 0.36120. 

9. Let us observe, all of them are greater than 0.2, which means that another round is necessary. Thus, go to 
the next point. 

10. The moderator informs the experts on the results and requests that each one reconsider the weights given 
to the criteria and evaluations. The process should be repeated in a second round. We must go to Step 5, 
but we will not repeat this for simplicity. See that if any of the projects had achieved Cl, < 0.2 it would 
not be taken into account for the next round and calculations and effort would simplify. Experts should 
concentrate more on reaching agreement on projects | and 3. 

11. We obtain P, = 2.6155, P, = 2.5880, P; = 4.8695, and P, = 3.1010. 

Thus, so far we have p3; > p4 > P; > pz and project pz is the preferred. Nevertheless, the moderator has 
to repeat the round. 


Conclusion 


This paper was devoted to introduce a method of evaluation and selection of scientific research proposals in 
academic or research institutions. The model is basically a Delphi method in a neutrosorphic environment. This 
method supports the research project selection according to a group of experts’ criteria. The Delphi method ensures 
that the opinion is agreed among the experts. The neutrosorphic framework offers the advantage of including not 
only uncertainty, but also indeterminacy in decision-making. Another advantage is that experts carry out 
evaluations with the help of linguistic scales, which makes the final results more veridical. In addition, the 
limitation of the classic Delphi method on slow convergence is attenuated, since with this model the projects that 
are reevaluated in the next round are only those where there was not sufficient consensus. To our knowledge, this 
is the first time that a model like this is designed which combines the Delphi method in a neutrosophic framework 
for the solution of this kind of problem. The paper provides the criteria to be followed for measurement, which 
does not mean that they are not modifiable. A hypothetical case study illustrates how to use the method and 
demonstrates its usefulness. 
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